Consistency and Generalization Bounds for Maximum Entropy Density Estimation
نویسندگان
چکیده
منابع مشابه
Consistency and Generalization Bounds for Maximum Entropy Density Estimation
We investigate the statistical properties of maximum entropy density estimation, both for the complete data case and the incomplete data case. We show that under certain assumptions, the generalization error can be bounded in terms of the complexity of the underlying feature functions. This allows us to establish the universal consistency of maximum entropy density estimation.
متن کاملMaximum Entropy Density Estimation with Incomplete Data
We propose a natural generalization of Regularized Maximum Entropy Density Estimation (maxent) to handle input data with unknown values. While standard approaches to handling missing data usually involve estimating the actual unknown values, then using the estimated, complete data as input, our method avoids the two-step process and handles unknown values directly in the maximum entropy formula...
متن کاملPerformance Guarantees for Regularized Maximum Entropy Density Estimation
We consider the problem of estimating an unknown probability distribution from samples using the principle of maximum entropy (maxent). To alleviate overfitting with a very large number of features, we propose applying the maxent principle with relaxed constraints on the expectations of the features. By convex duality, this turns out to be equivalent to finding the Gibbs distribution minimizing...
متن کاملMaximum Entropy Density Estimation and Modeling Geographic Distributions of Species
Maximum entropy (maxent) approach, formally equivalent to maximum likelihood, is a widely used density-estimation method. When input datasets are small, maxent is likely to overfit. Overfitting can be eliminated by various smoothing techniques, such as regularization and constraint relaxation, but theory explaining their properties is often missing or needs to be derived for each case separatel...
متن کاملNew Upper Bounds for Maximum - Entropy Sampling
We develop and experiment with new upper bounds for the constrained maximum-entropy sampling problem. Our partition bounds are based on Fischer's inequality. Further new upper bounds combine the use of Fischer's inequality with previously developed bounds. We demonstrate this in detail by using the partitioning idea to strengthen the spectral bounds of Ko, Lee and Queyranne and of Lee. Computat...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Entropy
سال: 2013
ISSN: 1099-4300
DOI: 10.3390/e15125439